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■ We consider tiie error due to a single bit-flip in a floating point number. We assume 

Q^, IEEE 754 double precision arithmetic, which encodes binary floating point numbers in a 

■^ I 64- bit word [1]. We assume that the bit-flip happens randomly so it has equi-probability 

ly-v ' (1/64) to hit any of the 64 bits. Since we want to mitigate the assumption on our initial 

floating-point number, we assume that it is uniformly picked among all normalized number. 

With this framework, we can summarize our findings as follows. 

"^ , The probability for a single bit flip to cause a relative error less than 10^^^ in 

^^ I a normalized floating-point number is above 25%; The probability for a single 

i/^ • bit flip to cause a relative error less than 10~® in a normalized floating-point 

number is above 50%; Etc. 
Similar studies could be done for binary floating point numbers in a 32-bit word, or 
^ I complex numbers; (so, s,c,d,z in LAPACK or BLAS notation); multiple bit-flips; or burst of 

CN ' bit-flips (that is multiple bit-flips which happen contiguously in the binary word). 

A 64-bit word consists of 3 fields: a sign bit (s), a biased 11 bit exponent (e), and a 52 



O 



(N 



Tjlj- I bit fraction (/). Excluding exceptions (to be explained) the encoded number is 

g : (-l)^(l./)22^-1023. 

^^ ' There are three exceptions: not a number (NAN), infinity (INF), and denormalized numbers. 

NANs are represented by the biased exponent of all ones and any nonzero fraction (sign does 
not matter). INFs are represented by the biased exponent of all ones and the zero fraction 
^ , (positive or negative infinity is possible). Denormalized numbers have a biased exponent of 

3 I all zeros. In this case the number represented is (— 1)'^(0./)22~^''^^. Zero is a special case 

of a denormalized number where the fraction is zero. We refer the reader to Overton [2] for 
more details on IEEE arithmetic. 

We only make one mild assumption on the initial floating-point number. For example, 
we assume it is a normalized number and then derive probabilities from that assumption. 
In that case, we assume that our initial floating-point number is randomly picked among 
all normalized number. In some cases, one could derive more information if one assumes 
more on the initial floating-point number. For example, if the number is 2.0e -|- 00, then the 
probability that a bit-flip changes it to INF is zero. That would be a different study. 

Let X be the original 64-bit word and x' be the 64-bit word after a single bit-flip between 
X and x' occurred. We will also denote the fields of x' as s' , e' , and /' when needed. Also, 
e{k) is the A;-th entry of the exponent field, and similar notation for other fields. 

We begin by calculating the probability that x' is normalized, NAN, INF, or denor- 
malized, given that x is normalized, NAN, INF, or denormalized. Table [1] summarizes the 
results. The narrative below explains where the results come from. 
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1. Let X be a normalized number. Therefore, the exponent is anything other than all 
ones or all zeros. For x' = NAN or x' = INF the exponent must be all ones. There are 
11 exponents corresponding to normalized numbers that are susceptible to becoming 
all ones due to a bit-flip (bit strings of all ones except 1 zero). The probability that 
the zero bit in the exponent is flipped is 1/64. The result will be an INF only if / = 0. 
Therefore we have 



Pr(normalized to NAN ) = ( -^^ j ( — j [l- -^] ~ 8.4005 x 10" 



and 



Pr(normalized to INF ) = (^-^ J (—) [ ^^ ) ~ 1-8653 x 10" 



20 



For x' to be a denormalized number the exponent must be all zeros. There are 11 
exponents corresponding to normalized numbers that are susceptible to becoming all 
zeros due to a bit-flip (bit strings of all zeros except 1 one). The probability that 
the one bit in the exponent is flipped is 1/64. For all factions the result will be a 
denormalized number. Therefore we have 



Pr(normalized to denormalized ) = ( -^^ ) ( 777 ) ~ 8.4005 x 10 
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Finally, we consider the probability that x' is a normalized number. If the exponent 
of X is not one of the 22 susceptible values then x' will be a normalized number, x' 
will also be a normalized number if the exponent is one of the susceptible exponents 
but the bit-flip is not in the corresponding entry. 



Pr(normalized to normalized ) = 1 — ( — r-j ) ( 777 ) ~ 9.9983 x 10 
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2. Let X = NAN. Therefore, e = (1 . . . 1)2 and / 7^ 0. x' will only be a normalized 
number if a bit in the exponent is flipped. Therefore, 

Pr(NAN to normalized) = — Ri 1.7188 x 10^^ 

There are 52 fractions that are susceptible to becoming all zeros (bit strings of all 
zeros except 1 one). The probability that the 1 bit is flipped is 1/64. If this occurs 
the result will be an INF, therefore 

Pr(NAN to INF) = ( -^ j ( — j Ri 1.8041 x 10"^^ 

A single bit-flip can not produce a denormalized number, therefore 

Pr(NAN to denormalized) = 0. 
The remaining cases will result in a NAN, therefore 

Pr(NAN to NAN) = 1 - ^ - (^) (^) « 8.2812 x 10"^ 



r -- — ____^ 


Normalized 


Denormalized 


NAN 


INF 


Normalized 


9.9983 X 10-^ 


8.4005 X 10-^ 


8.4005 X 10-5 


1.8653 X W-'^^> 


Denormalized 


1.7188 X 10"^ 


8.2812 X 10-^ 








NAN 


1.7188 X 10"^ 





8.2812 X 10-^ 


1.8041 X 10"^'' 


INF 


1.7188 X 10"^ 





8.1250 X IQ-^ 


1.5625 X 10-^ 



Table 1: Probability of a 64 bit word changing type due to a single bit-flip. 



3. Let X = INF. Therefore, e = (1 . . . 1)2 and / = (0 . . . 0)2- If the bit-flip occurs in /, 
then x' = NAN. If the bit-flip occurs in e, then x' is a normalized number. If the 
bit-flip occurs in s, then x' = iblNF. No bit-flip will cause a denormalized number. 
Therefore, we have the following probabilities: 



Pr(INF to NAN) = 52/64 ^ 8.1250 x 10"\ 
Pr(INF to normalized) = 11/64 « 1.7188 x 10 
Pr(INF to INF) = 1/64 « 1.5625 x 10"^ 
Pr(INF to denormalized) = 0. 
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4. Let X be a denormalized number. Therefore, e = (0...0)2. If the bit-flip occurs in 
either s or f then x' will also be a denormalized number. If the bit-flip occurs in e, 
then x' will be a normalized number. No bit-flip will cause an NAN or INF. Therefore, 
we have the following probabilities: 



Pr(denormalized to denormalized) = 53/64 ~ 8.2812 
Pr (denormalized to normalized) = 11/64 ~ 1.7188 x 
Pr(denormalized to NAN) = 0, 
Pr (denormalized to INF) = 0. 
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Relative error 

Normalized Numbers 

When X and x' are normalized or denormalized numbers we would like to calculate the 
relative error in x' as well as the probability that the relative error is below a given threshold. 
Let X and x' be normalized numbers. Then we have the following cases. 

1. If the flipped bit is in s, then x' = —x and 



x — x 



2. 



2. If the flipped bit is the k-th entry of /, then 

(0.0... 010... 0)22^"^°^^ 



x — X 



(1.0)2(2-'^)2 



fc\r,e-1023 



Hence, 



X-X'\ _ (1.0)2 .^-fcx 



1^1 (I-/) 

Since 1/2 < 1/(1. 7)2 < 1, we have 



2-fe-i ^ \xx^ ^ 2_fe^ 



3. If the flipped bit is in A;-th entry of e, then 

|x-x'| = |(l./)2(2-i023)(2-_2e')| 

= |(l./)2(2-^°23)(2e)[|i_2e'-^| 
= |x||l-2'^'-'^|. 

(a) If e{k) = 0, hence e' > e, then e' - e = 2^^"*^. 

(b) If e(/c) = 1, hence e' < e, then e' — e = —2^^"'^. 

Therefore, 

\x-x'\ _ f 22""' - 1 if x{k) = 0, 
\x\ ~\l- 2-2""' if x(fe) = 1. 

Note that 22""' - 1 > 1 and 1/2 < 1 - 2-2""' < 1 for fc = 1, . . . , 11. 



Let X be a normahzed number and x' be a denormahzed number. This can only happen 
if the bit-flip occurs in the exponent. The error is 



|(-l)^(l./)22-1023 _ (_i).(o,^)^2-^022[ ^ i^iii _ (^2^- 

IJ--/J2 



e| 



Since, < (0./)2/(l./)2 < 1/2 and e = 2"-*^ 

i<l-2-^<^f^<l. 
2 - \x\ ~ 

The fohowing theorem summarizes the results. 

Theorem 1. Let x is a normalized number. The relative error due to a single hit- flip is 

|x — x'|/|x| =2 if bit-flip in s, (1) 

2-^^-1 < |x - x'\/\x\ < 2-^ if bit-flip m f, (2) 

|x — x'|/|x| =2 — 1 > 1 if bit-flip in e and e{k) = 0, (3) 

1/2 < |x - x'\/\x\ = 1 - 2-2""' < 1 if bit-flip in e, e{k) = I, and e '/ (4) 

1/2 < 1 -2-2""' < |x-x'[/|2:| < 1 if bit- flip in e and e' = Q, (5) 

where k is the location of the bit- flip in the given field. 

Using these results, assuming x is a random normalized number, we can calculate the 
probability for the relative error to be in the following intervals 



1. Pr(|x - x'\/\x\ > 1) = 13/128 + (11/(2046)2^^2 (^^54) ^ 0.10156. This is mainly due 
to the probabihty of a bit-flip in s and in e (flipping to 1). See ([T]) and ([3]). The 
probabihty of each of these cases is 1/64 and 11/128, respectively. There is also a 
possibility for the relative error to be 1 if e' = (0 . . . 0)2 and / = 0, hence x' = 0. This 
is a special case of ([5]). This will occur with a probability of (ll/2046)(l/64)2^^^. 

2. Pr(l/2 < \x-x'\/\x\ < 1) = ll/128-((2i0-2)/2046)(l/64)-(ll/(2046))2-52(l/64) f» 
0.078133. This is mainly due to the probability of a bit-flip in e (flipping 1 to 0), which 
occurs with probability 11/128. See ^ and ([5]). However, these include the special 
case when the error is 1/2 and 1, which we do not include. The error will be 1/2 from 
this case if the first bit of the exponent is flipped, but the resulting number must still 
be normalized. This occurs with probability (2^'^ — 2)/2046)(l/64). The error will be 
1 for this case if x' = 0, which occurs with probability (ll/2046)(l/64)2~^2. 

3. Pr(|x - x'\/\x\ < 1/2) = 52/64 + (2^° - 2)(2046)(l/64) r^ 0.82030. The relative error 
will always be less than or equal to 1/2 if the bit-flip occurs in / (see ([21)), which occurs 
with probability 52/64. Including the special case from (j4]) such that the relative error 
is 1/2 gives the stated probability. 

4. Pr(|x — x'j/|x| < 2^*) = (53 — i)/64 for z = 2, . . . , 52. The relative error will always be 
less than or equal to 2~* if the bit-flip occurs in the j'-th entry of / (j > i). See ([2]). 
There are 53 — i entries of / greater than or equal to i. 

We can bound the probability that the error is less than some tolerance using the fourth 
case above. Below are the bounds on the probability for some error tolerances. 



Pr(|x- 


-xV\x 


< w-^) 


Pr(|x- 


-x'\/\x 


< 10-2) 


Pr(|x- 


-xy\x 


< 10-^) 


Pr(|x- 


-x'\/\x 


< 10-^) 


Pr(|x- 


-x'\/\x 


< 10-^) 


Pr(|x- 


-xV\x 


< 10-^) 


Pr(|x- 


-xV\x 


< 10-^) 


Pr(|x- 


-xV\x 


< 10-^) 



< 0.78125 

< 0.73438 

< 0.68750 

< 0.62500 

< 0.57812 

< 0.53125 

< 0.46875 

< 0.42188 



Pr(|x - x'\/\x\ < 10-^) < 0.37500 



Pr( 
Pr( 
Pr( 
Pr( 
Pr( 
Pr( 



\x — x' 
\x — x' 
\x — x' 
\x — X' 
\x — X 
\x — X 



< 10"^°) < 0.31250 

< 10"^^) < 0.26562 

< 10~^2) < 0.21875 

< 10"^^) < 0.15625 

< lO"^'') < 0.10938 

< 10"^^) < 0.06250 



Denormalized Numbers 

Let X be a denormalized number. If the bit-flip occurs in s, then x' is also a denormalized 
number and the error is the same as in the normalized case: 



X — X 



X 



2. 



If the bit-flip occurs in A;-th entry of /, then x' will again be a denormalized number and 

(0.0 . . . 010 . . . 0)22-i°22 ^ (i.o)2(2-'=)2-i°22^ 



X — X 



Hence, 



Since 1 < l/(0./)2 < 2^\ we have 



\x — X 



(1-0)2 ,^-k. 



2-fc ^ \xx^ ^ 2 
\x\ 



52-fc 



To improve these bounds we must be more specific on /. Let the first nonzero entry in / be 
the t-th entry. Then 2*-^ < 1/(0. /)2 < 2* and 



2t-k-i ^ \x x\ ^ ^t_i^^ 



Finally, if the bit-flip occurs in then k-th entry of e, then x' is a normalized number and 

|(0./)22-1022_(^_^)^2^'-1023| 



X — X 



2-i022((o./)2-2-i(l./)22^'| 



^1022 



-1022 



((0./)2 - (0./)22^ 
(0./)2(l-2^'| 



x||l-2" 
x||l-22 



For the last line we use e' = 2^^ . This is the same error as in the normalized case when 
the bit-flip in the exponent is from to 1. The following theorem summarizes the results. 

Theorem 2. Let x is a denormalized number. The relative error due to a single bit- flip is 

2 if bit- flip in s, 



\x — X \/\x\ 
2^-''-^ <\x- x'\/\x\ < 2*-^^ if bit-flip in f, 



\x — X 



1 > 1 if bit-flip in e, 



(6) 

(7) 
(8) 



where k is the location of the bit-flip in the given field and t is the location of the first nonzero 
entry in f. 
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